DATA PROCESSING CONTROL BASED ON USING A USABLE DISPLAY DEVICE

专利摘要:
---------------------- page 1---------------------- - 1/1 summary "Data processing control based on use of usable display device" techniques are described for controlling the operation of both a host device and a usable display device connected to the host device based on a usage situation of the usable display device. the techniques include the automatic determination of a usage situation for a usable display device based on feedback from one or more touch sensors inside the usable display device that indicates whether the usable display device is being used by a user. based on the determined usage situation, the usable display device controls its own operation (for example, controls the operation of the display screens the usable display device, a communication session with the host device, and the display processing of data received from the host device). the usable display device also sends an indication of the usage status to the host device. the host device then controls its own data processing for the usable display device based on the indicated usage situation.
公开号:BR112016004824A2
申请号:R112016004824
申请日:2014-08-05
公开日:2020-04-28
发明作者:Mohammad Rabii Khosro
申请人:Qualcomm Inc；
IPC主号:

专利说明:

DATA PROCESSING CONTROL BASED ON USING A USABLE DISPLAY DEVICE
Technical Field
[0001] The description refers to the processing of multimedia data and, more particularly, control over the processing of multimedia data.
Foundations
[0002] Wireless display systems (WD) include at least one host device and at least one client device that communicate over a wireless network. For example, a Direct Wi-Fi (WFD) system includes multiple devices communicating over a Wi-Fi network. The host device acts as a wireless access point and sends multimedia data, which can include video and audio data (AV), audio data and / or video data, for one or more client devices participating in a non-hierarchical group communication session (P2P) using one or more wireless communication standards, for example, IEEE 802.11. Multimedia data can be reproduced on both a host device monitor and monitors on each client device. More specifically, each of the participating client devices processes the received multimedia data for presentation on its display screen and audio equipment. Additionally, the host device can perform at least some processing of the multimedia data for presentation on the client devices.
[0003] The host device and one or more client devices can be wireless devices or wired devices with wireless communication capabilities. In one example, as wired devices, one or more of the host device and client devices may comprise televisions, monitors,
2/50 projectors, decoding boxes, DVD or Blu-Ray players, digital video recorders, laptop or desktop personal computers, video game consoles, and the like, which include wireless communication capabilities. In another example, as wireless devices, one or more of the host and client devices may comprise mobile phones, portable computers with wireless communication cards, personal digital assistants (PDAs), portable media devices, or other devices. flash memory with wireless communication capabilities, including so-called smartphones and smart pads or tablets, or other types of wireless communication devices (WCDs).
[0004] In some examples, at least one of the client devices may comprise a wearable display device. A wearable display device can comprise any type of wired or wireless display device that is used on a user's body. As an example, the wearable display device may comprise a head-worn monitor or a wireless head-mounted monitor (WHMD) that is worn on the user's head to position one or more display screens in front of the user's eyes . The host device is typically responsible for performing at least part of the processing of the multimedia data for display on the usable display device. In the case of wireless devices, both the host device and the usable display device can be powered by the limited resources of the battery. Improved battery life and conservation of battery life are therefore of utmost concern when designing WCDs and wireless display devices.
summary
3/50
[0005] In general, this description refers to techniques for controlling the operation of both a host device and a usable display device connected to the host device based on a usage situation of the usable display device. A usable display device typically includes a manual on / off switch and, when turned on, the usable display device can process data received from a host device for display on the usable display device. Conventionally, the host device processes and sends data to the usable display device, and the usable display device processes and displays received data regardless of whether the user is actually using the usable display device for visualization and interaction with displayed data. In the case of wireless devices, continuous processing is an unnecessary drain of relatively short battery life cycles for both the usable display device and the host device.
[0006] The techniques of this description include the automatic determination of a usable display device usage situation based on the return of one or more touch sensors from the usable display device that indicates whether the usable display device controls its own operation . For example, the usable display device can control the operation of the display screens of the usable display device, a communication session with the host device, and display the processing of data received from the host device. The wearable display device also sends an indication of the usage status to the host device. The host device can then control its own data processing for the display device
4/50 usable based on the indicated usage situation of the usable display device.
[0007] In one example, this description is described for a method of controlling a usable display device connected to a host device, the method comprising determining, with the usable display device, a usage situation of the usable display device based on the return of one or more touch sensors from the wearable display device indicating whether the wearable display device is used by a user, sending, with the wearable display device, an indication of the usage status of the display device usable display for the host device to control data processing on the host device for the usable display device, and control, with the usable display device, the operation of the usable display device based on the usage situation of the usable display device .
[0008] In another example, this description is directed to a method of controlling a host device connected to a usable display device, the method comprising receiving, with the host device, an indication of a situation of use of the device. usable display, where the usage status of the usable display device is determined based on the return of one or more touch sensors from the usable display device that indicates whether the usable display device is used by a user, and control, with the host device, from data processing on the host device to the usable display device based on the indicated usage situation of the usable display device.
5/50
[0009] In an additional example, this description addresses a usable display device connected to a host device, the usable display device comprising one or more touch sensors, and one or more processors configured to determine a usage situation for the usable display device based on feedback from touch sensors that indicates whether the usable display device is being used by a user, send an indication of the usage status of the usable display device to the host device to control data processing for the usable display device on the host device, and control the operation of the usable display device based on the usage situation of the usable display device.
[0010] In another example, this description is directed to a host device connected to a usable display device, the host device comprising one or more processors configured to receive an indication of a usable display device usage situation, where the situation Usage of the wearable display device is determined based on the return of one or more touch sensors from the usable display device that indicates whether the usable display device is used by a user, and control data processing for the display device usable based on the indicated usage situation of the usable display device.
[0011] In an additional example, this description is directed to a usable display device connected to a host device, the usable display device comprising means for determining a usable display device usage situation based on
6/50 on the return of one or more touch sensors from the usable display device indicating whether the usable display device is used by a user, means for sending an indication of the usage status of the usable display device to the host device to control the processing of data for the usable display device on the host device, and means to control the operation of the usable display device based on the usage situation of the usable display device.
[0012] In an additional example, this description is directed to a host device connected to a usable display device, the host device comprising means for receiving an indication of a usage situation from the usable display device, where the usage situation of the wearable display device is determined based on the return of one or more touch sensors from the wearable display device indicating whether the wearable display device is used by a user, and means to control the processing of data on the host device to the device usable display unit based on the indicated usage situation of the usable display device.
[0013] In another example, this description is directed to a computer-readable medium comprising instructions for controlling a usable display device connected to a host device, the instructions when executed causing one or more programmable processors to determine, with the device usable display, a usable display device usage situation based on the return of one or more touch sensors from the usable display device that indicates whether the usable display device is used by a user,
7/50 send, with the usable display device, an indication of the usage status of the usable display device to the host device to control data processing for the usable display device on the host device, and to control, with the usable display, operation of the usable display device based on the usage situation of the usable display device.
[0014] In an additional example, this description is intended for a computer-readable medium comprising instructions for controlling a host device connected to a usable display device, the instructions when executed cause one or more programmable processors to receive, with the device host, an indication of a usable display device usage situation, where the usable display device usage situation is determined based on the return of one or more touch sensors from the usable display device that indicates whether the display device usable is used by a user, and controls, with the host device, data processing for the usable display device based on the indicated usage situation of the usable display device.
[0015] Details of one or more examples of the description are presented in the attached drawings and in the description below. Other characteristics, objectives and advantages will be apparent from the description, drawings and claims.
Brief Description of Drawings
[0016] Figure 1 is a block diagram illustrating a Wireless Display (WD) system including a host device and a usable display device;
8/50
[0017] Figure 2 is a block diagram illustrating the host device and the usable display device of figure 1 in greater detail;
[0018] Figure 3 is a block diagram illustrating an example of a display device usable as a head mounted monitor (HMD) formed as glasses with touch sensors;
[0019] Figure 4 is a conceptual diagram illustrating an illustrative parallel plate capacitor;
[0020] Figure 5 is a circuit diagram illustrating an illustrative RC oscillator circuit including a touch sensor within the wearable display device of figure 3;
[0021] Figure 6 is a block diagram illustrating a location sensor unit included in the wearable display device of figure 2 in greater detail;
[0022] Figure 7 is a block diagram illustrating the host device of figure 2 in greater detail;
[0023] Figure 8 is a flow chart illustrating an illustrative operation of determining a usage situation for a usable monitor connected to a host device, and controlling the processing in the host device and usable display device based on the usage situation;
[0024] Figure 9 is a flow chart illustrating an operation illustrating the receipt of an indication of a situation of use of a display device usable on a host device, and controlling the processing on the host device based on the indicated use situation;
9/50
[0025] Figure 10 is a flow chart illustrating an illustrative operation of a location sensor unit included in a wireless head mounted display device (WHMD) and related control mechanisms of the WHMD device.
Detailed Description
[002 6] Figure 1 is a block diagram illustrating a Wireless Display (WD) system 10 including a host device 12 and a wearable display device 16. In the example in figure 1, the WD 10 system includes the host device 12 and only one client device, that is, the usable display device 16. In other examples, the WD 10 system may include additional client devices (not shown), which may comprise wearable display devices, wireless devices or devices wired with wireless communication capabilities.
[0027] In some examples, the WD 10 system can conform to the Wi-Fi Direct (WFD) standard defined by the Wi-Fi Alliance. The WFD standard allows communication between devices over Wi-Fi networks, that is, networks wireless local area, where devices negotiate their roles as access points or client devices. The WD 10 system can include one or more base stations (not shown) that support a plurality of wireless networks through which a non-hierarchical group communication session (P2P) can be established between host device 12, the display device usable 16, and other participating client devices. A communications service provider or other entity can operate centrally and manage one or more of these wireless networks using a base station as a network hub.
10/50
[0028] According to the WFD standard, the host device 12 can act as a wireless access point and receive a request from the usable display device 16 to establish a P2P group communication session. For example, host device 12 can establish the P2P group communication session between host device 12 and wearable display device 16 using the Real Time Sequencing Protocol (RTSP). The P2P group communication session can be established over a wireless network, such as a Wi-Fi network that uses a wireless communication standard, for example, IEEE 802.11a, 802.llg or 802.11η enhancements to standards Previous 802.11. Additional information regarding wireless networks can be found at Gast, M., 802.11® Wireless Networks: The Definitive Guide, O'Reilly, April 2002.
[002 9] Once the P2P group communication session is established, host device 12 can send multimedia data, which may include audio video (AV) data, audio data and / or video data, to the wearable display device 16, and any of the client devices, participating in the particular P2P group communication session. For example, host device 12 can send multimedia data to usable display device 16 using the Real Time Transport (RTP) protocol. The multimedia data can be reproduced on both a host device monitor 12 and display screens of the usable display device 16. For example, the usable display device 16 can process the media data received from the host device 12 for presentation on your display screens and audio equipment. Additionally, host device 12 can
11/50 perform at least some processing of the multimedia data for presentation on the usable display device 16.
[0030] A user of the wearable display device 16 can provide user registration via an interface, such as the human interface device (HID), included in or connected to the wearable display device 16. An HID can comprise one or more among a touch monitor, a recording device sensitive to a recording object (for example, a finger, a pen, etc.), a keyboard, a trackball, a mouse, a joystick, a remote control, a microphone, or similar. Usable execution device 16 sends the user record provided to the host device 12. In some examples, the usable display device 16 sends the user record through a reverse channel architecture referred to as a later user record channel ( UIBC). In this way, host device 12 can respond to the user record provided on the usable display device 16. For example, host device 12 can process the received user record and apply any effect of the user record to subsequent data sent to the device. usable display size 16.
[0031] The host device 12 can be a wireless device or a wired device with wireless communication capabilities. In one example, as a wired device, host device 12 may comprise one of a television, monitor, projector, set-top box, DVD or Bly-Ray player, digital video recorder, laptop or desktop personal computer, video game console, and the like, which includes wireless communication capabilities. And another
For example, as a wireless device, the host device 12 may comprise one of a mobile phone, a portable computer with a wireless communication card, a personal digital assistant (PDA), a portable media device, or other flash memory device with wireless communication capabilities, including the so-called smartphone and smart pad or tablet, or any other type of wireless communication device (WCD).
[0032] The wearable display device 16 can comprise any type of wired or wireless display device that is used on a user's body. As an example, the wearable display device 16 may comprise a head-worn monitor or a head-mounted monitor (HMD) that is worn on a user's head in order to position one or more display screens within the user's eyes . In general, the display screens of the wearable display device 16 may comprise one of a variety of display screens such as liquid crystal display (LCD), plasma display, organic light emitting diode (OLED) monitor , or any other type of display screen.
[0033] In one example, the wearable display device 16 may comprise an HMD device formed like glasses that include display screens on one or more of the lenses, and also include a bridge for the nose and extensions for the temples to be used in user's face. As another example, the wearable display device 16 may comprise an HMD device formed as glasses that include display screens in separate lenses or a single display screen, and which also include at least one strap to hold the glasses on the wearer's head. Although the wearable display device 16 is basically described in this description as being an HMD, in other examples the
13/50 wearable display device 16 may comprise display devices that are used on other parts of the user's body, such as on the user's neck, shoulders, arm or wrist. Specific examples of HMDs and their operation are described in more detail in Rolland, J. & Há, H., Head-Mounted Display Systems, Encyclopedia of Optical Engineering, 2005.
[0034] In the WD 10 system, the host device 12 and the wearable display device 16 are typically wireless devices. For example, the wearable display device 16 may comprise a wireless HMD (WHMD) that connects wirelessly to the host device 12, and the host device 12 may comprise a WCD, such as a mobile smartphone or smart pad. In this example, in addition to typical WCD operations, the host device 12 performs at least some multimedia data processing for presentation of the usable display device 16 and the processing of the user record from the interactivity of the user interface on the display device. wearable 16. The host device 12 can perform these operations with a power manager originating from a rechargeable battery that is limited by size and weight to fit within the frame of a portable device.
[0035] The power manager and battery of the wearable display device 16 can even be further limited due to the fact that the wearable display device 16 is intended for use on the user's body. Since the wearable display device 16 can be an HMD worn on the user's head, the structure of the wearable display device 16 needs to be small and light enough to remain comfortable during use. These size and weight restrictions can result in
14/50 relatively small batteries being included in the wearable display device 16 compared to other mobile devices. The wearable display device 16, therefore, may need to perform multimedia data processing for presentation and interactivity of the user interface with a power manager originating from a rechargeable battery that is limited by size, weight, balance, temperature restrictions and health.
The WFD standard does not provide some of the power management protocols for devices, such as the host device 12, which operates as the access points, that is, the Opportunistic Energy Saving protocol and the Absence Notice protocol. Both power management protocols allow a device to operate as an access point to save energy by going into sleep mode during convenient or pre-planned periods, without dismantling a P2P group communication session with one or more client devices. More information regarding these WFD power management protocols is available from CampsMur, D., et al., Designing Energy Efficient Access Points with Wi-Fi Direct, The International Journal of Computer and Telecommunications Networking, Volume 55, Issue 13, September of 2011.
[0037] The wearable display device 16 may include a manual on / off switch (not shown) and, when turned on, the wearable display device 16 processes the data received from the host device 12 for display on the usable display device 16. On simply the wearable display device 16, however, does not indicate whether a user is actually using the wearable display device 16 for viewing and
15/50 interaction with the displayed data. Conventionally, a host device will process and send data to a usable display device, and the usable display device will process and display received data regardless of whether the user is actually using the usable display device. In the case of wireless devices, continuous processing is an unnecessary drain on the cycle or relatively short battery life of both the usable display device and the host device.
[0038] The wearable display device 16 necessarily requires a user to use the device, so that operation of the usable display device 16 and processing of related multimedia data on the host device 12 is only necessary when the user is in fact using the device. Since the user must use the wearable display device 16, the use of the wearable display device 16 can be intrusive and interfere with the user's normal activities. The use of the usable display and device 16, therefore, can be stopped arbitrarily, and it is unlikely that the user will remember to manually turn off the usable display device 16.
[0039] In general, this description refers to the operation control techniques of both the host device 12 and the wearable display device 16 connected to the host device 12 based on a usage situation, that is, if it is in use or not, of the usable display device 16. According to the techniques, the usage situation of the usable display device 16 is automatically detected to minimize unnecessary processing and conserve cycle = battery life on both device
16/50 host 12 and the wearable display device 16 without depending on user interaction. As illustrated in figure 1, the wearable display device 16 includes a location sensor unit 20 configured to automatically determine whether the wearable display device 16 is being used by a user for viewing and / or interacting with the displayed data.
[0040] The techniques of this description include the use of the wearable display device 16 including one or more touch sensors (not shown in figure 1) positioned in locations that are in contact with or close to the user when the user is using the display device wearable 16. In an example where the wearable display device 16 comprises a WHMD device formed like glasses, the wearable display device 16 may include at least one sensor on a nose bridge and at least two sensors on the extensions of the temples that will come into contact. contact with the wearer's nose and ears, respectively, when the glasses are worn. In this way, touch sensors will inevitably come into contact with the user when the user is using and using the wearable display device. In other examples, the wearable display device 16 may include more or less touch sensors positioned at different locations depending on the shape or shape of the device. In addition, in some cases, the wearable display device 16 may include touch sensors capable of being triggered by proximity to the user's body without requiring actual contact with the user's body.
[0041] According to the techniques, the location sensor unit 20 automatically determines a usage situation of the usable display device 16 based on the feedback from the touch sensors of the detection device.
17/50 usable display 16. The return indicates to the location sensor unit 20 if the usable display device 16 is being used by the user. Based on the determined usage situation, the wearable display device 16 controls its own operation. For example, the wearable display device 16 can control the operation of one or more display screens of the wearable display device 16, the communication session with the host device 12, the processing of displaying data received from the host device 12. The wearable display device 16 also sends an indication of the usage situation to the host device 12. The host device 12 can then control its own data processing for the wearable display device 16 based on the indicated usage situation of the usable display device. 16.
[0042] Figure 2 is a block diagram illustrating the host device 12 and the wearable display device 16 of figure 1 in greater detail. For the purposes of this description, the host device 12 and the wearable display device 16 will basically be described as wireless devices with battery size and weight limitations, resulting in a short battery life. For example, host device 12 can comprise a smartphone or smart pad, or other portable WCD, and wearable display device 16 can comprise a WHMD device. In other respects, however, host device 12 and wearable display device 16 may comprise wired devices or wired devices with wireless communication capabilities.
[0043] In the example illustrated in figure 2, host device 12 includes a processor for
18/50 application 30, a system interrupt processor 34, a wireless controller 36, a connection processor 38, a multimedia processor 42 and a monitor 44. The application processor 30 includes a user registration processor (UI ) 32. In other examples, host device 12 may comprise additional functional units or modules used to control and perform WCD operations. As an example, a more detailed version of the host device 12 is described below with reference to figure 7.
[0044] As illustrated in figure 2, the wearable display device 16 includes the location sensor unit 20, the wireless controller 46, the connection processor 48, the controller 50, the multimedia processor 52, display screens 54 and touch sensors 56. Controllers 50 comprise a master controller for the wearable display device 16, and controls the general operation of the wearable display device 16. The location sensor unit 20 and touch sensors 56 of the wearable display device 16 and its operation according to the techniques of this description are described in greater detail below and in relation to figures 3 to 6.
[0045] In general, the host device 12 processes the multimedia data for presentation on its own monitor 44, and can also process the multimedia data for presentation on the usable display device 16. Additionally, the usable display device 16 can receive the user record through an interface, such as an HID, and can send the user record to host device 12 for processing. In figure 2, the transfer of both multimedia data and the user registration between the
19/50 host device 12 and wearable display device 16 is illustrated as a path 62.
[0046] To transfer media data from the host device 12 to the usable display device 16, path 62 can start on application processor 30. Application processor 30 provides an environment in which a variety of applications can run on the host device 12. Illustrative applications include text applications, e-mail applications, video or slide show applications, presentation applications, video conferencing applications, and the like. The application processor 30 can receive data for use by these applications from the internal or external storage location and / or internal or external sensors or cameras associated with the host device 12. Applications running on the application processor 30, in turn , generate multimedia data for presentation to a user of the host device 12 and / or usable display device 16. In other examples, the path 62 may start at the multimedia processor 42 or some other functional device that generates the multimedia data or receives multimedia data directly from storage locations and / or sensors or cameras.
[0047] The multimedia processor 42 can display the received multimedia data process for display on the monitor 44 of the host device 12. Additionally, the multimedia processor 42 can process the received multimedia data for transmission and presentation on the usable display device. 16. In the latter case, the wireless controller 36 packages the processed data for transmission. The packaging of the processed data may include the grouping of data into packages, frames or cells that may depend on the pattern of
20/50 wireless communication used over the Wi-Fi network 40. The connection processor 38 then transmits the processed data to the usable display device 16 using the Wi-Fi network 40. The connection processor 38 manages the device connections host 12, including a P2P group communication session with the usable display device 16 over the Wi-Fi network 40, and the transmission and reception of data over the connections.
[0048] The transfer of multimedia data continues along path 62 on the wearable display device 16 when connection processor 48 receives data transmitted from host devices 12. Similar to connection processor 38 on host device 12, the processor connection 48 of the wearable display device 16 manages the connections of the wearable display device 16, including a P2P group communication session with the host device 12 over Wi-Fi network 40, and the transmission and reception of data over the connections . The wireless controller 46 unpacks the received data for processing by the multimedia processor 52. The multimedia processor 52 then displays the processes that the data received for presentation on the display screens 54 of the usable display device 16.
[0049] To transfer the user record from the usable display device 16 to the host device 12, the path 62 can be followed, in reverse order with respect to what was described above, starting with the multimedia processor 52. The multimedia 52 can receive user registration via an HID or other user interface (not shown) included in or connected to the usable display device 16. Wireless controller 46 packages the user registry, and connection processor 48 transmits the register
21/50 user packaged over Wi-Fi network 40 for host device 12. On host device 12, ο connection processor 38 receives the transmitted user record, and wireless controller 36 unpacks the received user record for processing by the multimedia processor 42 and the processor UI 32. In this way, the host device 12 can respond to the user record by applying any effect of the user record on data processing on the media processor 42 and / or applications running on the processor. application 30.
[0050] Conventionally, the host device 12 and the usable display device 16 continue to operate as described above until some user interaction occurs to disconnect, put into latent state, or de-energize the usable display device 16. Processing of data to be displayed on the usable display device 16 regardless of whether the user is using the usable display device 16, however, it consumes the substantial energy resources of both the host device 12 and the usable display device 16. To conserve the battery life or cycle, the techniques of that description include the location sensor unit 20 and touch sensors 56 on the wearable display device 16 to allow an automatic determination of a usage situation for the usable display device 16, this, and if the wearable display device 16 is used by a user for viewing and / or interacting with the displayed data. Additionally, the techniques include notifying the host device 12 of the usage status of the wearable display device 16. Thus, the techniques allow the wearable display device 16 to enter
22/50 automatically in a reduced power state, in which all components except the location sensor unit 20 are turned off, without relying on user interaction to disconnect, put into latent state or de-energize the usable display device 16. The techniques also allow host device 12 to disable data processing on host device 12 for wearable display device 16 when wearable display device 16 is not in use.
[0051] The location sensor unit 20 of the wearable display device 16 is designed to be always operational even when the remaining components of the wearable display device 16 are in a latent state or turned off. In some cases, a portion of the controller 50 responsible for operating the location sensor unit 20 may also remain energized. In order to always stay on, the location sensor unit 20 is designed to consume ultra low energy, for example, approximately 10 microwatts (pW). In addition, the location sensor unit 20 may require additional minimal hardware on the wearable display device 16. The location sensor unit 20 can also prevent user registration controls that would unnecessarily engage host device 12 and can be used for some application-specific UI controls on the wearable display device 16 to minimize latency.
[0052] The location sensor unit 20 receives feedback from the touch sensors 56 which indicates whether the wearable display device 16 is being used by the user. Based on the feedback, the location sensor unit 20 continuously determines the usage status of the usable display device 16. As
23/50 described in more detail below, in some cases, the location sensor unit 20 can generate an oscillation frequency that changes based on whether the touch sensors 56 are in contact with the user's body, and determine the usage situation of the wearable display device 16 based on a comparison of the generated oscillation frequency and a limit frequency value.
[0053] The touch sensors 56 can be positioned inside the wearable display device 16 in locations that will be in contact with or close to the user when the user is using the wearable display device 16. An example in which the wearable display device 16 comprising a WHMD device formed as glasses is described in greater detail with reference to figure 3. In some cases, each of the touch sensors 56 may comprise a capacitance touch sensor that increases an oscillation frequency generated by the location sensor unit 20. In this example, when the oscillation frequency generated by the location sensor unit 20 is
greater than that a value in frequency limit , the unity in location sensor 20 determines that the device in exhibition usable 16 is in use. [0054] When a change in situation in
utilization occurs, for example, a user places or removes the wearable display device 16, the location sensor unit 20 can inform the controller 50 about the usage situation determined through a processor interrupt request 58. In other examples, the location sensor unit 20 can continuously send usage status indications to controller 50 regardless of whether a change in usage status has occurred. Controller 50, in turn,
24/50 time, it can generate a virtual processor interrupt request 60 to indicate the usage status of the usable display device 16 for the host device 12. As illustrated in figure 2, the virtual processor interrupt request 60 is packaged by wireless controller 46 and transmitted by connection processor 48 over Wi-Fi network 40 to host device 12. On host device 12, connection processor 38 receives the transmitted virtual processor interrupt request 60, and the wireless controller 36 unpacks the user record received for processing by the system interrupt processor 34 and the application processor 30.
[0055] In the case where the wearable display device 16 is in the reduced power state and a user puts the wearable display device to use, the location sensor unit 20 receives feedback from the touch sensors 56 indicating that the Usable display 16 is being used by the user. Based on the feedback, the location sensor unit 20 determines whether the wearable display device 16 is being used, and indicates the intended use for controller 50. For example, the location sensor unit 20 can send the interrupt request from direct processor 58 to controller 50 to wake up or activate the other components of the usable display device 16. Controller 50 controls the operation of the usable display device 16 based on the indication of the usage status of the usable display device 16. For For example, controller 50 may instruct connection processor 48 to establish a communication session with host device 12. Additionally, controller 50 may allow display processing on the data processor.
25/50 multimedia 52 of the data received from the host device 12, and activate the display screens 54 of the usable display device 16 in order to display the processed data.
[0056] Upon receipt of the indication from the location sensor unit 20 that the wearable display device 16 is in use, controller 50 also sends an indication that the wearable display device is in use to the host device 12. For example, controller 50 may send virtual processor interrupt request 60 to host device 12. Application processor 30 from host device 12 controls the processing of data on host device 12 to usable display device 16 based on indication of the usage status of the usable display device 16. For example, the application processor 30 may allow data processing in the multimedia processor 42 for transmission and display in the usable display device 16. In some cases, the application processor 30 can also instruct the connection processor 38 to establish the communication session with the usable display device 16, and transmitting the processed data to the wearable display device 16 based on the indication that the wearable display device 16 is in use. In addition, application processor 30 may allow processor UI 32 to process any user registration received from the usable display device 16, and adjust application processing and data processing based on the received usage record.
[0057] In the case where the wearable display device 16 is in use and a user removes the wearable display device, the location sensor unit 20
26/50 receives feedback from touch sensors 56 indicating that the wearable display device 16 is not being used by the user. Based on the feedback, the location sensor unit 20 determines that the wearable display device 16 is no longer in use, and indicates the intended use for controller 50. For example, the location sensor unit 20 can send the request direct processor interrupt switch 58 for controller 50 to put it to sleep, turn it off or otherwise disable the other components of the usable display device 16. Controller 50 controls the operation of the usable display device 16 based in indicating the usage status of the usable display device 16. For example, controller 50 can disable display processing in multimedia processor 52 of data received from host device 12, and disable display screens 54 of the usable display device 16. Controller 50 may also instruct connection processor 48 to dismantle the communication session with host device 12.
[0058] After receiving the indication from the location sensor unit 20 that the wearable display device 16 is not in use, controller 50 also sends an indication that the wearable display device is not in use to the host device 12. For example, controller 50 can send virtual processor interrupt request 60 to host device 12. Application processor 30 from host device 12 controls data processing on host device 12 to usable display device 16 with based on the indication of the usage status of the wearable display device 16. For example, the
27/50 application processor 30 can disable data processing on multimedia processor 42 for transmission and display on the usable display device 16. In some cases, application processor 30 may also instruct connection processor 38 to dismantle the session of communication with the wearable display device 16, and terminate data transmission to the wearable display device 16 based on the indication that the wearable display device 16 is in use. In addition, application processor 30 can disable processor UI 32 from processing any user registration received from the usable display device 16. In this way, the techniques in this description can improve battery life or cycle and can reduce unnecessary processing data on both the wearable display device 16 and the host device 12.
[0059] Figure 3 is a block diagram illustrating an example of the display device usable 16 as an HMD formed as glasses with the touch sensors 56A-56C (touch sensors 56). As illustrated in figure 2 described above, the wearable display device 16 includes wireless controller 46 that prepares data for transmission using the P2P group communication session with host device 12 established over Wi-Fi network 40, the controller 50 that controls the operation of the wearable display device 16, and the multimedia processor 52 that performs the display processing of the data received from the host device 12. In the illustrated example, the spectacle lenses comprise the display screens 54 for which the multimedia processor 52 processes the video data for presentation to the user. Additionally, the display device usable
28/50 includes speakers 64A and 64B (speakers 64) for which the multimedia processor 52 processes the audio data for presentation to the user.
[0060] As illustrated in figure 3, the wearable display device 16, formed as glasses, includes the display screens 54 on the lenses held together by a nose bridge 63, and temple extensions 65A and 65B (temple extensions 65 ) that allow the wearable display device 16 to be used on a user's face. In this example, the touch sensors 56 are positioned in locations that are inevitably contacted by the user's body when the user is using the wearable monitor device 16. In the illustrated example, the wearable display device 16 includes a touch sensor 56C positioned on the eyeglass nose bridge and touch sensors 56A and 56B positioned on the eyeglass temples that will be in contact with the wearer's nose and ears, respectively, when the glasses are worn. In other cases, the wearable display device 16 may include touch sensors capable of being triggered by proximity to the user's body without requiring actual contact with the user's body. In that case, the touch sensors can be positioned in locations on the wearable display device 16 that will be at least close to the user's body, but not in physical contact with the user's body.
[0061] The location sensor unit 20 of the wearable display device 16 includes a touch transducer 66 and a touch detector 68. Touch transducer 66 is connected directly to each of the touch sensors 56 to receive feedback from touch sensors 56. Touch transducer 66 converts the touch feedback from touch sensors 56 into electrical feedback. In cases where the location sensor unit 20 generates a
29/50 oscillation frequency to determine the usage situation of the wearable display device 16, the touch transducer 66 can convert the return of the touch sensors 56 into additional capacitance which causes the generated oscillation frequency to increase when the touch 56 are in contact with the user's body.
[0062] The touch detector 68 receives the converted feedback from the touch transducer 66 which indicates whether one or more touch sensors 56 are in contact with the user body, and determines whether the wearable display device 16 is in use based on on return. More specifically, the touch detector 68 can compare the oscillation frequency generated based on the return of the touch sensors 56 with a predetermined frequency value. For example, when the generated oscillation frequency is greater than the threshold frequency value, the touch detector 68 can determine that the wearable display device 16 is being used by the user. The touch detector 68 can then send a processor interrupt request directly to the controller 50 to indicate the determined usage situation of the usable display device 16.
[0063] In the illustrated example, the wearable display device 16 includes three touch sensors 56. In other examples, the wearable display device 16 may include more or less touch sensors. In some cases, it may be advantageous to use two or more of the touch sensors 56 so that the location sensor unit 20 can detect that all touch sensors 56 are in contact with or close to the user and the wearable display device 16 is being used properly, or if less than all touch sensors 56 are in contact with the user and the wearable display device 16 is
30/50 positioned incorrectly or is being held on one or more touch sensors 56. For example, the location sensor unit 20 will generate the highest oscillation frequency when all touch sensors 56 are simultaneously in contact with a surface of the user's body, indicating that the user is using the wearable display device 16. The threshold frequency value can be a predetermined value that requires all touch sensors 56 to be in contact with the user. In other examples, the threshold frequency value can be a predetermined value that requires at least one of the touch sensors 56 to be in contact with the user.
[0064] As described above, the location sensor unit 20 can be designed to be always on. The touch transducer 66, therefore, can continuously receive the feedback from the touch sensors 56 and convert the feedback to the location sensor unit 20 to generate a constant update oscillation frequency. In addition, the touch detector 68 can continuously compare the updated oscillation frequency with the limit frequency value to determine a current usage situation of the usable display device 16.
[00065] In some cases, the touch detector 68 sends a processor interrupt request directly to the controller 50 to indicate the usage situation only when a change occurs in the determined usage situation of the usable display device 16. Thus, controller 50 is only notified of the usage situation when a wake up or shut down operation needs to be performed. In other cases, the touch detector 68 continuously sends an indication of the usage situation to controller 50 and controller 50
31/50 then detects when a change in the usage situation has occurred for the control operation of the usable display device 16, and sends an indication of a change in the usage situation to the host device 12. In any case, the determination of the usage situation use, and subsequent wake-up or shut-down operation can be performed as a background process of the wearable display device 16.
[0066] In the illustrated example, the wearable display device 16 is an HMD formed like glasses. In other examples, the wearable display device 16 can comprise any type of wired or wireless display device that is used on the user's body, including HMD with a different form factor than shown in figure 3. As an example, the device wearable display 16 may comprise an HMD device formed as glasses which includes the display screens in separate lenses or a single display screen and which also includes at least one strap to secure the glasses to the wearer's head. As some examples, the wearable display device 16 may comprise a display device that is used on other parts of the user's body, such as on the user's neck or shoulders.
[0067] Figure 4 is a conceptual diagram illustrating an illustrative parallel plate capacitor 70. According to the techniques of that description, parallel plate capacitor 70 can be associated with one of the touch sensors 56 included in a usable display device 16 of figure 3. Capacitor 70 includes an upper plate 72A and a lower plate 72B (plates 72) positioned in parallel with respect to each other, and a dielectric material 74 interspersed between plates 72A and 72B. In figure 4, the dielectric material 74 is indicated as having a
32/50 real permittivity equal to the product of the relative permissiveness, E _r , of the dielectric material 74 and the permittivity of free space, E _r . The permissiveness of the dielectric material 74 indicates the ability of the dielectric material 74 to transmit an electric field.
[0068] In general, the capacitance of the parallel plate capacitor 70 indicates the capacity of capacitor 70 to store an electrical charge. The capacitance of the parallel plate capacitor 70 depends on the area of the plates 72, the distance between the plates 72 and the relative or constant permittivity of the dielectric material 74 between the plates 72. Specifically, the capacitance of the capacitor of
parallel board 70 is equal to C = ε ₀ * ε _Γ * (A / d), where A represents the area of the plates 72 and d represents the distance between the plates 72. [0069] Figure 5 it is a diagram of circuit illustrating a oscillator circuit Illustrative RC 75
including a 56A touch sensor within the wearable display device 16 of figure 13. In some instances, the RC 75 oscillator circuit can be considered a relaxing oscillator. The RC 75 oscillator circuit includes an amplifier that generates an oscillation frequency based on the selective frequency register provided by an RC network, which includes at least one resistor (R) and at least one capacitor (C).
[0070] In the example shown in figure 5, the RC 75 oscillator circuit also includes the touch sensor 56A of the wearable display device 16 of figure 3. The touch sensor 56A can comprise a capacitance touch sensor that includes a plate or electrode that is positioned inside the wearable display device 16 so that the touch sensor 5 6A contacts the
33/50 user when the wearable display device 16 is used. When the 5 6A touch sensor is in contact with the user's body, a capacitor is created in which the user's skin acts as a dielectric material and the Earth acts as a ground for the 56A touch sensor electrode.
[0071] In an example illustrated in figure 5, when the touch sensor 5 6A is not in contact with a surface of the user's body, current 78 does not flow to the touch sensor 56A and the frequency of oscillation generated depends only on R and C. This oscillation frequency can be considered the baseline or standard oscillator frequency of the RC 75 oscillator circuit. In another example illustrated in figure 5, when the 56A touch sensor is in contact with the user's body, current 7 6 flows to the touch sensor 56A and through the user body to the ground. In this case, the capacitance of the user body, for example, Ctouch, is added to the RC network. The additional capacitance changes the general RC time constant of the RC network and changes the generated oscillation frequency. The techniques of this description use the generated oscillation frequency value to determine whether the wearable display device 16 is used by the user.
[0072] Figure 6 is a block diagram illustrating the location sensor unit 20 included in the wearable display device 16 of figure 2 in greater detail. As illustrated in figure 2 described above, the wearable display device 16 includes wireless controller 46 which prepares data for transmission using the P2P group communication session with host device 12 established over Wi-Fi network 40, controller 50 which performs the display processing of the data received from the host device 12 for presentation on the display screens 54. Additionally, as
34/50 illustrated in figure 3 described above, the location sensor unit 20 of the wearable display device 16 includes the touch transducer 66 which receives feedback from the touch sensors 56 and the touch detector 68 which determines a usage situation of the display device usable based on the feedback converted by the touch transducer 66.
[0073] In the example shown in figure 6, the location sensor unit 20 additionally includes an RC oscillation circuit that generates an oscillating frequency based on feedback from the touch sensors 56. The RC oscillator circuit can operate substantially similar to the RC 75 oscillator circuit in figure 5 with the inclusion of additional capacitance touch sensors. The touch sensors 56 are illustrated in figure 6 as additional capacitors included in the RC oscillator circuit of the location sensor unit 20 that are connected to a ground through a user's body. Each of the touch sensors 56 can operate substantially similar to the touch sensor 56A described with reference to figure 5.
[0074] When the wearable display device 16 is first energized, the location sensor unit 20 activates a grounding circuit 69 to ground all touch sensors 56 for a predetermined period of time. During this period, the RC oscillator circuit generates a standard oscillation frequency for the wearable display device 16 when touch sensors 56 are in contact with the user's body. Once the standard oscillation frequency is determined, the location sensor unit 20 can initiate the usage situation determination operation.
35/50
[0075] The touch transducer 66 receives the feedback from the touch sensors 56 during a scan timer period and the RC oscillator circuit generates an oscillation frequency based on the feedback. The scan timer period can be a predetermined period of time during which the RC oscillator circuit of the location sensor unit 20 generates the oscillation frequency based on the return of the touch sensors 56. The scan timer period can allow that the resulting oscillation frequency stabilizes before the touch detector 68 compares the oscillation frequency with a threshold frequency value to determine a usage situation for the usable display device 16.
[0076] When one or more of the touch sensors 56 are in contact with the user's body, the touch transducer 66 receives feedback as a faster capacitance discharge rate through the additional capacitors. This return of the touch sensors 56 results in the RC oscillator circuit generating a higher oscillation frequency than when the touch sensors 56 are not touched. The touch detector 68 then compares the highest oscillation frequency with the threshold frequency value to determine whether the frequency is high enough to indicate that the user is using the wearable display device 16 for use.
[0077] For example, the limit frequency value can be a predetermined value that is lower than the highest oscillation frequency, but higher than the oscillation frequency generated when none of the touch sensors 5 6 is in contact with the user's body. In some cases, the threshold frequency value can be predetermined so that the touch detector 68 can only be
36/50 determine that the wearable display device 16 is in use when all touch sensors 56 are in contact with the user. In other cases, the threshold frequency value can be predetermined so that the touch detector 68 determines that the wearable display device 16 is in use when at least one of the touch sensors 56 is in contact with the user.
[0078] As illustrated in figure 6, the wearable display device 16 also includes a power manager 79 that can store battery status information that reflects whether the wearable display device 16 is plugged into an outlet or using its power reserve. battery, and if using battery backup, the remaining battery power level. In some cases, battery status information may be displayed to the user of the wearable display device 16, for example, using a small battery icon, lights or sounds to indicate different battery conditions. Power manager 79 can update battery status information almost continuously to reflect an accurate battery situation for the user of the usable display device 16. In some cases, when the battery reserve is below a minimum value, the power manager 79 can initiate a shutdown or input operation in mode 1 watch out for the usable display device 16 regardless of its usage situation.
[007 9] Figure 7 is a block diagram illustrating the host device 12 of figure 2 in greater detail. In the illustrated example, host device 12 includes application processor 30 with UI processor 32, system interrupt processor 34, wireless controller 36, connection processor 38, multimedia processor 42, monitor 44, the memory
37/50 external 80, local memory 82, general purpose graphics processing unit (GPGPU) 84, application data manager 86, display processor 88, ο battery monitoring system 90 and security manager 92.
[0080] In general, application processor 30, UI processor 32, system interrupt processor 34, wireless controller 36, connection processor 38, multimedia processor 42 operate as described above with respect to the figure 2. Applications running on application processor 30 generate multimedia data, for example, AV data, video data, or audio data, for presentation to a user of the host device 12 and / or the usable display device 16 or some another client device connected to the host device 12. In some cases, the multimedia processor 42 can process the same video data to display on both monitor 44 and the external monitor of the usable display device 16 or another client device. In other cases, the multimedia processor 42 can process the video data for display on only one of the monitor 44 and an external monitor.
[0081] To present the data on the host device 12, the multimedia processor 42 can perform some pre-processing, and the display processor 88 performs the processing of the display of the video data for presentation on the monitor 44. In the case of audio, the multimedia processor 42 can again perform some pre-processing, and an audio processor (not shown) can perform additional audio processing for presentation on one or more speakers (not shown) of the host device 12. To present the data on the wearable display device 16 or
38/50 some other client device connected to host device 12, multimedia processor 42 can perform some pre-processing, and wireless controller 36 and connection processor 38 then package and transmit, respectively, the processed data to the client device over Wi-Fi 40. Connection processor 38 manages connections from host device 12 over Wi-Fi 40. In other examples, connection processor 38 can manage a 3G or 4G modem connection, a global positioning system (GPS) connection, and / or a Bluetooth connection.
[0082] In some cases, data stored in external memory 80 can be received from an external storage device, such as a flash drive, through a peripheral interface, for example, a universal serial bus (USB) interface or a secure digital card (SD) interface. Data stored in external memory 80 can also be received from the store or in real time from a private network or a public network, for example, the Internet, via connection processor 38. The application data manager 86 can move the data for applications from external memory 80 and local memory 82 for easy access by application processor 30. Additionally, GPGPU 84 can perform any graphics processing for video game applications or other applications that require 3D representations.
[0083] The host device 12 also includes the battery monitoring system 90 which monitors a battery status of the host device 12. The battery monitoring system 90 can store battery status information that reflects whether the host device 12 is connected to an outlet or are using your
39/50 battery backup, and if using battery backup, the remaining battery power level. In some cases, the battery status information may be displayed to the user of the host device 12, for example, using a small battery icon, lights or sounds to indicate different battery conditions. The battery monitoring system 90 can update the battery status information almost continuously to reflect an accurate battery status for the user of the host device 12.
[0084] The components of the host device 12 illustrated in figure 7 are merely illustrative. In other examples, the host device 12 may include more, less and / or different components. Host device components 12 can be implemented as any one of a variety of suitable circuit packs, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable port sets in field (FPGAs), discrete logic, software, hardware, firmware or any combination thereof. The monitor 44 on the host device 12 can comprise one of a variety of display devices such as a liquid crystal display (LCD), a plasma monitor, an organic light emitting diode (OLED) monitor, or other type of display. display screen.
[0085] External memory 80 and local memory 82 on host device 12 may comprise any of a wide variety of volatile and non-volatile memory, including, but not limited to random access memory (RAM) such as random access memory synchronized dynamics (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM),
40/50 electrically programmable and erasable read-only memory (EEPROM), FLASH memories, and the like. External memory 80 and local memory 82 may comprise a computer-readable storage medium for storing media data, in addition to other types of data. External memory 80 and local memory 82 additionally store instructions and program code that are executed by application processor 30 and / or multimedia processor 42 as part of carrying out the techniques described in that description.
[0086] Figure 8 is a flow chart illustrating an illustrative operation of determining a usage situation for a usable display device (WDD) connected to a host device, and controlling the processing on the host device and the usable display device based on in the usage situation. The illustrative operation is described with respect to the wearable display device 16 connected to the host device 12 of figures 1 and 2.
[0087] The WDD 16 location sensor unit 20 determines a WDD 16 usage situation based on the feedback received from one or more touch sensors 56
included in the WDD 16 (100). The sensors touch 56 may be positioned at the WDD 16 in places that will come into contact or what They are close to the user when the user are using WDD 1 6. In some examples, WDD 16
comprises a wireless head mounted display device (WHMD) formed like glasses, as shown in figure 3, including at least one of the touch sensors, for example, the 56C touch sensor, located on an eyeglass bridge and at least two of the touch sensors, for example, touch sensors 56A and 56B, located on the temple extensions.
41/50
[0088] The location sensor unit 20 may include an oscillator circuit that uses a combination of resistors and capacitors to generate an oscillation frequency. In this example, each of the touch sensors 56 connected to the WDD 16 adds capacitance to the oscillating circuit. When one or more of the touch sensors 56 are in contact with a surface of the user's body (for example, the user's head or face), the return of the touch sensors 56 comprises a higher capacitance discharge rate through the additional capacitors , which results in the oscillating circuit generating a higher oscillation frequency than when touch sensors 56 are not touched. In this example, the location sensor unit 20 determines the usage situation for WDD 16 by generating an oscillation frequency based on the feedback from the touch sensors 56 and comparing the resulting oscillation frequency with a threshold frequency value to determine if the user is using WDD 16.
[0089] When the oscillation frequency is higher than the limit frequency value, the location sensor unit 20 determines that WDD 16 is in use. Conversely, when the oscillating frequency is less than or equal to the limit frequency value, the location sensor unit 20 determines that WDD 16 is not in use. The location sensor unit 20 will generate the highest oscillation frequency when all touch sensors 56 are simultaneously in contact with a surface of the user's body, indicating that the user is using WDD 16 for use. The limit frequency value, therefore, can be a predetermined value that is lower than the highest oscillating frequency, but greater than a frequency of oscillation generated when none of the touch sensors 56
42/50 are in contact with a user body surface. In some cases, the threshold frequency value can be predetermined so that the location sensor unit 20 determines only that WDD 16 is in use when all touch sensors 56 are in contact with the user. In other cases, the threshold frequency value can be predetermined so that the location sensor unit 20 determines that WDD 16 is in use when at least one of the touch sensors 56 is in contact with the user.
[0090] In some cases, the location sensor unit 20 can continuously determine the usage situation of WDD 16, and, at fixed intervals or by determining a change in the usage situation, send a direct processor interrupt request for controller 50 of WDD 16 indicating the usage situation of WDD 16. Controller 50, in turn, can send a virtual processor interrupt request to host device 12 indicating the usage status of WDD 16. As an example , when the location sensor unit 20 determines that WDD 16 is in use (SIM branch 102), controller 50 of WDD 16 sends an indication that WDD 16 is in use to host device 12 to allow data processing on host device 12 to display on WDD 16 (104).
[0091] The WDD 16 controller 50 also controls its own operation based on the WDD 16 usage situation. For example, when the location sensor unit 20 determines that WDD 16 is in use (SIM branch 102), the controller 50 of WDD 16 can establish a communication session, for example, the non-hierarchical wireless connection (P2P), with the device
43/50 host 12 (106). In addition, controller 50 of WDD 16 can activate display screens 54 of WDD 16 (108). The controller 50 of the WDD 16 can also allow display processing by the multimedia processor 52 of data received from the host device 12 for display on the WDD 16 (110). WDD 16 and host device 12 can continue to operate in this state of full power until the location sensor unit 20 determines that WDD 16 is no longer in use by the user.
[0092] As another example, when the location sensor unit 20 determines that WDD 16 is not in use (branch NOT 102), controller 50 of WDD 16 sends an indication that WDD 16 is not in use to the device host 12 to disable data processing on host device 12 for display on WDD 16 (112). Controller 50 of WDD 16 also controls its own operation based on the usage situation of WDD 16. For example, when the location sensor unit 20 determines that WDD 16 is not in use (branch NO 102), WDD 16 can go into a state of reduced energy. In that case, controller 50 of WDD 16 may disable display processing by the multimedia processor 52 of data received from host device 12 for display on WDD 16 (114). In addition, controller 50 of WDD 16 can disable display screens 54 of WDD 16 (116). Controller 50 of WDD 16 can also dismantle a communication session, for example, a non-hierarchical wireless connection (P2P), with host device 12 (118). WDD 16 and host device 12 can continue to operate in that reduced power state until the location sensor unit 20 determines that WDD 16 is in use.
44/50
[0093] Figure 9 is a flow chart illustrating an illustrative operation of receiving an indication of a situation of use of a usable display device (WDD) on a host device, and control of processing on the host device based on the situation of use indicated. The illustrative operation is described with respect to the host device 12 connected to the wearable display device 16 of figures 1 and 2.
[0094] Host device 12 receives an indication of a WDD 16 usage situation from controller 50 of WDD 16 (120). As described above with reference to figure 8, the location sensor unit 20 of the WDD 16 determines the usage situation of the WDD 16 based on the feedback received from one or more touch sensors 56 included in the WDD 16 that indicate whether a user is using WDD 16, and indicates the usage situation for WDD 16 controller 50 using a direct processor interrupt request. In some cases, application processor 30 from host device 12 receives a virtual processor interrupt request from controller 50 of WDD 16 indicating the usage status of WDD 16. Application processor 30 from host device 12 can receive requests from interruption of virtual processor indicating the usage situation of WDD 16 at fixed intervals or by changing the usage situation of WDD 16.
[0095] Application processor 30 on host device 12 controls data processing on host device 12 for WDD 16 based on the usage situation of WDD 16. In some cases, application processor 30 can also control the operation of a communication session, for example, a non-hierarchical wireless connection (P2P), with WDD 16 and
45/50 data transmission to WDD 16 via the communication session based on the indicated usage situation of WDD 16. As an example, when host device 12 receives an indication that WDD 16 is in use (SIM branch 122 ), the application processor 30 allows the data processor by the multimedia processor 42 of the host device 12 to display on WDD 16 (124). The host device 12 can continue to operate in this state of full power until the application processor 30 of the host device 12 receives an indication that WDD 16 is no longer in use by the user.
[0096] As another example, when host device 12 receives an indication that WDD 16 is not in use (branch NO 122), application processor 30 disables data processing by multimedia processor 42 of host device 12 to display on WDD 16 (126). In addition, application processor 30 can generate a message to the user of host device 12 and WDD 16 that WDD 16 has entered a reduced power state (128). In some examples, the generated message can be presented to the user on the monitor 44 of the host device 12. In this way, the user is notified that WDD 16 has not been in use for some predetermined period of time, and is automatically entering the state reduced energy. The host device 12 can continue to operate in this reduced power state until the application processor 30 of the host device 12 receives an indication that WDD 16 is in use by the user.
[0097] Figure 10 is a flow chart illustrating an illustrative operation of a location sensor unit included in a wireless head mounted monitor (WHMD) device and control mechanisms
46/50 of the WHMD device. The illustrative operation is described with respect to the wearable display device 16 as WHMD 16 including the location sensor unit 20 and the controller 50 from figure 2.
[0098] Starting with a wake up in WHMD 16 (140), the unity in sensor in location 20 of WHMD 16 receive the comeback From sensor s of touch 56 inside WHMD 16
during a scan timer period. The wake-up mechanism can be a user manually turning WHMD 16. The scan timer period can be a predetermined period of time during which the location sensor unit 20 generates an oscillation frequency based on the return of the touch sensors 56. The scan timer period can allow the resulting oscillation frequency to stabilize before the location sensor unit 20 performs a determination of the WHMD 16 usage situation.
[0099] When the scan timer expires (SIM branch 142), the location sensor unit 20 determines whether WHMD 16 is in place, that is, used by a user (144). When WHMD 16 is in place (SIM branch 144), location sensor unit 20 sends a processor interrupt request directly to the WHMD 16 controller 50 to the wake 50 controller and other WHMD 16 components. If WHMD 16 is not connected by P2P to the host device 12 (branch NOT 150), the controller 50 can initiate the establishment of a communication session, for example, a P2P group, with the host device 12 (152). Once WHMD 16 is connected via P2P to host device 12 (SIM branch 150), controller 50 sends a virtual interrupt request to host device 12 indicating that WHMD
47/50 is in use to allow data processing on host device 12 for WHMD 16. Controller 50 can then control the operation of WHMD 16 in a full power state, as described above with respect to figure 8. QHMD 16 can operate in a full power state until the location sensor unit 20 determines that WHMD 16
is not being May s used by the user (144 ). [0100] When WHMD 16 does not it is on site (branch NOT 144), the unit in sensor in location 20 directs the controller 50 for start one timer in connection for WHMD 16 (14 6). 0 period in timer in disconnection can be a period predetermined in time during which the controller 50 turn off WHMD 16 before dismantling the connection without thread
P2P with host device 12. For example, before the disconnect timer expires (branch NO 146), controller 50 can reduce or minimize WHMD 16 display processing of AV data received from host device 12 until the screens 54 are disabled and the sound is muted (154). For example, controller 50 can reduce the quality of service (QoS) of data created for display. Controller 50 also sends a virtual interrupt request to host device 12 indicating that WHMD 16 is not in use to disable data processing on host device 12 for WHMD 16. After the disconnect timer has expired (branch SIM 146), controller 50 can initiate cancellation of the wireless P2P connection, that is, a P2P connection energy saving mode, between WHMD 16 and host device 12 (148).
[0101] Once controller 50 disables the processing of data display in WHMD 16 (branch
48/50
SIM 156) and dismantle the wireless P2P connection (148), the WHMD 16 enters a reduced power state, as described above with respect to figure 8 until the location sensor unit 20 determines that the WHMD 16 is being used by the user. In addition, based on the indication of the controller 50, the host device 12 disables data processing on the host device 12 for WHMD 16 and generates user messages related to the reduced energy state of WHMD 16 (158). WHMD 16 can operate in the reduced power state until the location sensor unit 20 determines that WHMD 16 is being used by the user (144).
[0102] In one or more examples, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored in or transmitted as one or more instructions or code in a computer-readable medium. The computer-readable medium may include a computer data storage medium or communication medium including any medium that facilitates the transfer of a computer program from one place to another. In some instances, the computer-readable medium may comprise a non-transitory, computer-readable medium. The data storage medium can be any available media that can be accessed by one or more computers or one or more processors to retrieve the instructions, code and / or data structures for implementing the techniques described in that description.
[0103] By way of example, and not by limitation, such a computer-readable medium may comprise non-transitory medium such as RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage
49/50 magnetic, or other magnetic storage devices, flash memory, or any other means that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is properly called a computer-readable medium. For example if the software is transmitted from a network site, a server or another remote source using a coaxial cable, a fiber optic cable, a twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies, such as infrared, radio and microwave are included in the definition of medium. Floppy disk and disk, as used here include compact disk (CD), laser disk, optical disk, digital versatile disk (DVD), floppy disk, and blu-ray disk, where floppy disks normally reproduce data magnetically, while disks reproduce data optically with lasers. Combinations of the above must also be included in the scope of computer-readable medium.
[0104] The code can be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic sets (FPGAs), or another set of equivalent integrated or discrete logic circuits. Accordingly, the term processor, as used here, may refer to any one of the above structure or any other structure suitable for implementing the techniques described here. Additionally, in some respects, the functionality described here can be provided within dedicated hardware and / or
50/50 software configured for encoding and decoding, or incorporated into a combined codec. In addition, the techniques can be fully implemented in one or more circuits or logic elements.
[0105] The techniques of this description can be implemented on a wide variety of devices or devices, including a wireless device, an integrated circuit (IC) or a set of ICs (for example, a chip set). Various components, modules or units are described in this description to emphasize the functional aspects of the devices configured to perform the described techniques, but do not necessarily require realization by different hardware units. Instead, as described above, multiple units can be combined into one codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with appropriate software and / or firmware.
[0106] Various embodiments of the invention have been described. These and other modalities are within the scope of the following claims.

权利要求:
Claims (15)
[1]
1. Method of controlling a usable display device connected to a wireless host device, the method characterized by the fact that it comprises:
determine, with the wearable display device, a wearable display device usage situation based on the return of one or more touch sensors from the usable display device indicating whether the wearable display device is being used by a user;
send, with the usable display device, an indication of the usage status of the usable display device to the host device;
enable or disable, with the usable display device, data processing performed on the host device to generate multimedia data for presentation on the usable display device, based on the indication of the usage status of the usable display device; and control, with the usable display device, operation of the usable display device based on the usage situation of the usable display device.
[2]
2. Method of controlling a wireless host device connected to a usable display device, the method characterized by the fact that it comprises:
receive, with the host device, an indication of a usable display device usage situation, where the usable display device usage situation is determined on the usable display device based on the return of one or more touch sensors from the device usable display that indicates whether the usable display device is used by a user; and enable or disable, with the host device, data processing performed on the device
Petition 870170054320, of 7/31/2017, p. 5/12
2/6 host to generate multimedia data for presentation on the usable display device based on the indicated usage situation of the usable display device.
[3]
3. Usable display device for connection to a wireless host device, the usable display device characterized by the fact that it comprises:
one or more touch sensors; and one or more processors configured to determine a usable display device usage situation based on feedback from the touch sensors indicating whether the usable display device is being used by a user, send an indication of the usage status of the device usable display for the host device, enable or disable data processing performed on the wireless host device to generate multimedia data for presentation on the usable display device based on the indication of the usage status of the usable display device, and control device operation usable display device based on the usage situation of the usable display device.
[4]
4. Usable display device according to claim 3, characterized by the fact that one or more processors are configured to:
generate an oscillation frequency based on the return of the touch sensors from the usable display device, where the oscillation frequency changes when the touch sensors are in contact with the user; and determining whether the wearable display device is in use or not, based on a comparison of the oscillation frequency and a threshold frequency value.
[5]
5. Usable display device according to claim 3, characterized by the fact that one or more processors are configured to determine
Petition 870170054320, of 7/31/2017, p. 6/12
3/6 continuously the usage status of the usable display device, and generate a direct processor interrupt request based on a change in the usage situation.
[6]
6. Usable display device, according to claim 3, characterized by the fact that one or more processors are configured to control the operation of one or more display screens of the usable display device, a communication session with the host device , and display the processing of data received from the host device.
[7]
7. Usable display device, according to claim 3, characterized by the fact that, based on the usable display device in use, the one or more processors are configured for one or more among establishing a communication session with the device host, activate one or more display screens of the usable display device, enable display processing of data received from the host device, or request that a processor on the host device enable data processing on the host device to generate multimedia data for presentation on the usable display device; and / or in which, based on the usable display device not being in use, the one or more processors are configured for one or more of them to enter a reduced power state, disable display processing of data received from the host device, disable one or more display screens of the usable display device, or end a communication session with the host device.
Petition 870170054320, of 7/31/2017, p. 7/12
4/6
[8]
8. Usable display device according to claim 7, characterized by the fact that the one or more processors are configured to:
start a disconnect timer;
before the disconnection timer expires,
reduce processing in display of received data of host device up until that the screens display are disabled; and through the expiration of timer in disconnect, shut down The session communication with The host device.
[9]
9. Usable display device according to claim 3, characterized in that, based on the usable display device not being used, the one or more processors are configured to request that a processor from the host device disable processing data on the host device to not generate multimedia data for presentation on the usable display device.
[10]
10. Wearable display device according to claim 3, characterized in that the wearable display device comprises a wireless head mounted display device (WHMD) formed as glasses that includes at least one of the touch sensors located on an eyeglass bridge and at least two touch sensors located on the eyeglass temple extensions.
[11]
11. Host device for connection to a usable display device, the host device characterized by the fact that it comprises:
a memory configured to store data; and one or more processors connected to the memory and configured to receive an indication of a
Petition 870170054320, of 7/31/2017, p. 12/8
5/6 use of the usable display device, where the usage situation of the usable display device is determined on the usable display device based on the return of one or more touch sensors from the usable display device that indicates whether the display device usable is being used by a user, and enable and disable data processing performed on the host device to generate multimedia data for presentation on the usable display device based on the indicated usage situation of the usable display device.
[12]
Host device according to claim 11, characterized by the fact that one or more processors are configured to control the operation of a communication session with the usable display device and data transmission to the usable display device based on in the indicated use situation of the usable display device.
[13]
13. Host device according to claim 11, characterized by the fact that, based on an indication that the usable display device is in use, the one or more processors are configured to enable data processing on the host device for generate multimedia for presentation on the usable display device; and / or disable data processing on the host device to avoid generating multimedia data for presentation on the usable display device.
[14]
14. Hosting device according to claim 13, characterized by the fact that, based on the indication that the usable display device is not in use, the one or more processors are configured to generate a message to the user that the wearable display device has entered a reduced power state.
Petition 870170054320, of 7/31/2017, p. 9/12
6/6
[15]
15. Computer readable memory characterized by the fact that comprising instructions stored therein, the instructions being executable by a computer to perform the method of controlling a usable display device connected to a wireless host device as defined in any one of claims 1 or 2.

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112016004824A2|2020-04-28|DATA PROCESSING CONTROL BASED ON USING A USABLE DISPLAY DEVICE

---

US10674451B2|2020-06-02|Power sensitive wireless communication radio management

---

LiKamWa et al.2014|Draining our glass: An energy and heat characterization of google glass

---

EP2778945B1|2017-08-23|Systems, methods, and apparatuses for handling timeouts

---

KR20150081419A|2015-07-14|Systems, methods, and apparatuses for synchronizing port entry into a low power state

---

US20150378409A1|2015-12-31|USB Power Delivery Controller Sharing

---

KR20130131425A|2013-12-03|Methods and apparatus for saving power

---

US11256639B2|2022-02-22|Method, apparatus, system to enhance the device policy manager | to manage an expended USB-C ecosystem

---

US20180189222A1|2018-07-05|APPARATUSES AND METHODS FOR MULTILANE UNIVERSAL SERIAL BUS | COMMUNICATION OVER EMBEDDED UNIVERSAL SERIAL BUS |

---

US20180060261A1|2018-03-01|Apparatuses to resolve roles for dual role serial bus devices

---

WO2017000247A1|2017-01-05|Method for terminal and network device interaction and terminal

---

RU2664398C2|2018-08-17|Architecture with ultra low capacity to support the permanently turned on access to memory

---

US20190121647A1|2019-04-25|Autonomous host detection

---

KR20170057668A|2017-05-25|Electronic system and electronic device

---

US20150007190A1|2015-01-01|Techniques to aggregate compute, memory and input/output resources across devices

---

US10133324B2|2018-11-20|Thermal mitigation user experience

---

TWI674506B|2019-10-11|Mirroring of screens

---

TW201602912A|2016-01-16|Electronic apparatus and control method thereof

---

TW201602780A|2016-01-16|Electronic apparatus and control method thereof

---

JP2013222420A|2013-10-28|Peripheral device server

---

TW201338591A|2013-09-16|Method for power saving and communication apparatus using the same

同族专利:

---

公开号 | 公开日

---

CN105518615B|2017-06-30|

---

KR101751657B1|2017-06-27|

---

JP6022740B1|2016-11-09|

---

CA2920708C|2019-02-19|

---

CA2920708A1|2015-03-12|

---

US20150062022A1|2015-03-05|

---

KR20160032269A|2016-03-23|

---

US9183612B2|2015-11-10|

---

EP3042279A1|2016-07-13|

---

WO2015034617A1|2015-03-12|

---

US9772494B2|2017-09-26|

---

JP2016536684A|2016-11-24|

---

CN105518615A|2016-04-20|

---

US20160025975A1|2016-01-28|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

JPH02205151A|1989-02-03|1990-08-15|Nec Corp|Communication equipment|

---

US7596242B2|1995-06-07|2009-09-29|Automotive Technologies International, Inc.|Image processing for vehicular applications|

---

JPH07294844A|1994-04-22|1995-11-10|Canon Inc|Display device|

---

JPH1065996A|1996-08-23|1998-03-06|Olympus Optical Co Ltd|Head wearable display device|

---

US7474276B2|2000-06-20|2009-01-06|Olympus Optical Co., Ltd.|Display system and microdisplay apparatus|

---

JP3988632B2|2002-11-28|2007-10-10|日本電気株式会社|Control method of glasses-type display|

---

JP2004236242A|2003-01-31|2004-08-19|Nikon Corp|Head mounted display|

---

JP4426918B2|2004-07-20|2010-03-03|オリンパス株式会社|Information display system|

---

EP2046026A4|2006-07-25|2010-05-05|Nikon Corp|Outputting apparatus and image display apparatus|

---

JP2009288892A|2008-05-27|2009-12-10|Nikon Corp|Head mount display|

---

WO2011097564A1|2010-02-05|2011-08-11|Kopin Corporation|Touch sensor for controlling eyewear|

---

WO2011098854A1|2010-02-11|2011-08-18|Varidal Company Limited|Method for the detection of a body with respect to a surface, detecting device for the implementation of the method, and surface comprising such devise|

---

US20120249797A1|2010-02-28|2012-10-04|Osterhout Group, Inc.|Head-worn adaptive display|

---

JP5512884B2|2010-07-20|2014-06-04|エンパイアテクノロジーディベロップメントエルエルシー|Augmented reality proximity sensing|

---

US8954177B2|2011-06-01|2015-02-10|Apple Inc.|Controlling operation of a media device based upon whether a presentation device is currently being worn by a user|

---

US8184067B1|2011-07-20|2012-05-22|Google Inc.|Nose bridge sensor|

---

JP5953714B2|2011-11-24|2016-07-20|セイコーエプソン株式会社|Device, head-mounted display device, device control method, and head-mounted display device control method|

---

JP2013114124A|2011-11-30|2013-06-10|Seiko Epson Corp|Image display device, power supply control method in image display device, and power supply control program in image display device|

---

JP2012191624A|2012-03-21|2012-10-04|Nikon Corp|Head-mounted display device|

---

US9183612B2|2013-09-04|2015-11-10|Qualcomm Incorporated|Wearable display device use-based data processing control|US9183612B2|2013-09-04|2015-11-10|Qualcomm Incorporated|Wearable display device use-based data processing control|

---

JP6338344B2|2013-10-04|2018-06-06|キヤノン株式会社|COMMUNICATION DEVICE, COMMUNICATION DEVICE CONTROL METHOD, AND PROGRAM|

---

US9986044B2|2013-10-21|2018-05-29|Huawei Technologies Co., Ltd.|Multi-screen interaction method, devices, and system|

---

US9223451B1|2013-10-25|2015-12-29|Google Inc.|Active capacitive sensing on an HMD|

---

KR102233728B1|2013-10-31|2021-03-30|삼성전자주식회사|Method, apparatus and computer readable recording medium for controlling on an electronic device|

---

GB2520025B|2013-11-06|2021-04-14|Arm Ip Ltd|Calibrating proximity detection for a wearable processing device|

---

KR102244855B1|2014-04-11|2021-04-27|삼성전자 주식회사|Method for choosing input and output device and electronic device implementing the same|

---

US10649500B2|2014-07-21|2020-05-12|Beam Authentic, Inc.|Centralized content distribution in a wearable display device network|

---

KR102216126B1|2014-07-31|2021-02-16|삼성전자주식회사|Wearable device for operating using vein authentication and operation method of the same|

---

US10375081B2|2014-08-13|2019-08-06|Intel Corporation|Techniques and system for extended authentication|

---

KR20160069936A|2014-12-09|2016-06-17|삼성전자주식회사|Method for controlling power saving and electronic device implementing the same|

---

KR20160090582A|2015-01-22|2016-08-01|삼성전자주식회사|Smart cards and Method of manufacturing the smart card|

---

KR102281341B1|2015-01-26|2021-07-23|엘지전자 주식회사|Method for controlling source device at sink device and apparatus for the same|

---

US9794684B2|2015-02-13|2017-10-17|Seiko Epson Corporation|Head-mounted display device, control method for the same, and computer program|

---

US10083018B1|2015-03-09|2018-09-25|Fitbit, Inc.|Systems and methods for distributing application templates to wearable client devices|

---

CN107430273A|2015-03-12|2017-12-01|埃西勒国际通用光学公司|Method for customizing installing type sensor device|

---

CN104703148A|2015-03-16|2015-06-10|安徽华米信息科技有限公司|Information feedback method and device|

---

KR20160113903A|2015-03-23|2016-10-04|엘지전자 주식회사|The mobile terminal and control method thereof|

---

EP3093194B1|2015-04-24|2021-09-22|Ricoh Company, Ltd.|Information provision device|

---

CN104851368A|2015-06-04|2015-08-19|京东方科技集团股份有限公司|Flexible display device|

---

JP2017030530A|2015-07-31|2017-02-09|スズキ株式会社|Image display system|

---

KR20170051013A|2015-11-02|2017-05-11|엘지전자 주식회사|Tethering type head mounted display and method for controlling the same|

---

US10827320B2|2015-12-04|2020-11-03|LenovoPte. Ltd.|Presentation of information based on whether user is in physical contact with device|

---

US10289196B2|2016-07-01|2019-05-14|North Inc.|Techniques for ocular control|

---

CN106648491A|2016-10-12|2017-05-10|深圳市优景观复光电有限公司|Interactive LED display apparatus and display method|

---

WO2018123170A1|2016-12-27|2018-07-05|三井化学株式会社|Sensor module, temple, frame and eyewear|

---

US20180297540A1|2017-04-17|2018-10-18|Paul R. Juhasz|Interactive skin for vehicle|

---

US10571691B1|2017-08-09|2020-02-25|Facebook Technologies, Llc|Service loop for a head-mounted display|

---

TWI659352B|2018-05-15|2019-05-11|元太科技工業股份有限公司|Touch sensor, electronic paper display panel and electronic paper display apparatus|

---

WO2020185447A1|2019-03-08|2020-09-17|Facebook Technologies, Llc|Systems and methods for dynamic scheduling|

---

US20210333866A1|2020-04-28|2021-10-28|Facebook Technologies, Llc|Interface between host processor and wirelss processor for artificial reality|

法律状态:
2018-11-06| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|

---

2020-04-22| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

---

2021-10-19| B350| Update of information on the portal [chapter 15.35 patent gazette]|

优先权:

---

申请号 | 申请日 | 专利标题

---

US14/018,131|US9183612B2|2013-09-04|2013-09-04|Wearable display device use-based data processing control|

---

PCT/US2014/049814|WO2015034617A1|2013-09-04|2014-08-05|Wearable display device use-based data processing control|

---